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Overview 


In recent years, interest has spiked in data-driven decision making in education — that is, using various types of student 
data to inform decisions in schools and classrooms. The increased emphasis on data use is based on the belief that student 
data can be an important lever for improved teaching and learning. Specifically, the more information educators have about 
students, the better they will be able to understand and address students’ academic strengths and needs. As a result, many 
schools, school districts, and states have invested in resources and tools designed to provide teachers, principals, and other 
key stakeholders with access to student data throughout the school year. 

In October 2008, the Council of the Great City Schools and American Institutes for Research (AIR) launched a project 
funded by The Bill & Melinda Gates Foundation that focused on understanding the use of data as a lever for instructional 
improvement. This study focused specifically on interim (also known as benchmark) assessments that are often adopted at 
the district level. Interim assessments are typically characterized as falling between regular (often daily) formative assess- 
ments and annual summative assessments. They are administered systematically at regular intervals (e.g., every six to eight 
weeks) throughout the school year in order to gather information about students’ knowledge and skills. Interim assessments 
are described as serving several purposes, including predicting student performance on end-of-year state tests, diagnosing 
student strengths and weaknesses on topics taught during a specific time frame, and providing data that can be used to evalu- 
ate a curriculum or instructional program. 

The goals of this project were to (1) document and understand current interim assessment data-use practices in urban 
school districts and (2) to test the li nks between data-use practices and perceptions and student achievement. This brief is 
a summary of the report that focused on the second objective: examining the empirical relationships between teacher- and 
school-level data use and student achievement in mathematics and reading. By examining the extent to which certain data- 
use practices are related to student achievement, the study expands on the existing body of literature on the use of interim 
assessments to drive instructional improvement. The full report. Charting Success: Data Use and Student Achievement in 
Urban Schools can be found at www.cgcs.org. along with its companion pieces Using Data to Improve Instruction in the 
Great City Schools: Documenting Current Practices, and Using Data to Improve Instruction in the Great City Schools: Key 
Dimensions of Practice. 

Key Dimensions of Data Use 

To guide the study, we formulated a theory of action that hypothesized how data practices at multiple levels ( district, school, 
principal, teacher, and student) may be related to each other and ultimately to improved student achievement. We began by 
acknowledging that interim assessment data may be used for three general purposes: 

• To better understand the academic needs of individual students and respond to these needs by targeting 
instruction, support, and resources accordingly. 

• To better understand the instructional strengths and weaknesses of individual teachers and use this infor- 
mation to focus professional development, peer support, and improvement efforts. 

• To support and facilitate conversations among teachers and instructional leaders regarding strategies for 
improving instruction. 

These practices, in turn, are thought to lead to improved and more responsive teaching and therefore yield increased student 
achievement. 
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At the outset, we identified four key dimensions of interim assessment data use — Context, Supports for Data Use, Working 
with Data, and Instructional Responses. This proposed theory of action is shown in Exhibit 1 . The diagram flows from left 
to right, with key dimensions on the left leading, in theory, to key dimensions to the right. 

In brief, the context factors at the far left are hypothesized to lead to the establishment of supports for data use, which in 
turn are expected to facilitate teachers and principals working with data. Working with data hypothetically leads to a change 
in teaching strategies or instructional responses, which ultimately leads to improved student achievement. Barriers to using 
data are represented with one bar below the key dimensions with dashed arrows signifying that real or perceived barriers 
can interrupt data use at any point in the process. 


Exhibit 1. Using Data from Interim Assessments to Improve Student Achievement 
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These concepts in the theory of action are composed of a number of practices and perceptions related to data use at all lev- 
els of the education system. For the empirical study linking the key dimensions with student achievement, we focused on 
classroom-level data-use practices and perceptions by surveying teachers, and we focused on school-level data-use practices 
and perceptions by surveying principals. 

Key Dimension 1: Context 

The first key dimension broadly encompasses the various contextual and cultural factors that may be related to data use. 
Key elements of Context include the assessment context, the instructional context, the state and district data culture, and 
the school data culture. Although other contextual elements are theoretically relevant (e.g., the political or the economic 
context), our theory and measurement of data use focused on factors that we hypothesized are most relevant to the use of 
data in school districts, schools, and classrooms. 

Key Dimension 2: Supports for Data Use 

This dimension involves the specific elements of practice related to logistical and operational support for using data, includ- 
ing the data infrastructure, organizational supports, time allocation, and staffing and human resources necessary to support 
using interim assessment data to guide and improve instruction. 

Key Dimension 3: Working with Data 

This dimension focuses on the ways that teachers and principals work individually and together to understand student data, 
including individual teacher attention to data, collaboration around data, and making sense of data, which refers to specific 
ways of reviewing assessment data to understand student performance. 

Improved Knowledge. Implicit in the path from making sense of data to responding in the classroom is a change in educa- 
tors’ knowledge about student needs and principal and district knowledge about teacher and school needs. 

Important aspects of Improved Teacher Knowledge include improved awareness and understanding of the following: 

• Instructional needs and challenges of individual students 

• Instructional needs and challenges facing their classrooms as a whole 

• Teachers’ own strengths and weaknesses 

• Strategies and resources for addressing the needs of struggling students 

• Strategies for teaching and re-teaching specific concepts in different ways 

Important aspects of Improved Principal and District Knowledge include improved awareness and understanding of the 
following: 


• Instructional needs and challenges facing individual classrooms or teachers and the school as a whole 

• Teachers’ (and schools’) strengths and weaknesses 

• Strategies and resources for addressing the needs of teachers and schools 

Although we identify teacher and principal knowledge explicitly in the theory of action, measuring their knowledge was 
beyond the scope of this study. 
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Key Dimension 4: Instructional Responses 

Instructional Responses are the ways that schools and teachers translate the improved knowledge they glean from review- 
ing interim assessment results and use it to change classroom-level instruction. This dimension also includes actions (e.g., 
interventions, professional development) implemented at the school and district levels in response to patterns and trends in 
student assessment data. 

Barriers to Data Use 

As shown in Exhibit 1, perceived Barriers to Data Use associated with any of the aforementioned key dimensions may 
disrupt the theoretical progression toward improved student achievement. The most common Barriers identified in previous 
research include a lack of time to engage in data exploration and reflection, poor assessment quality or validity, lack of data 
accuracy, lack of alignment with standards and pacing, lack of timeliness and accessibility of data, limited staff capacity, and 
negative perceptions of the use of interim assessment data for teacher evaluation. 


Research Design 

The overarching goal of the study was to understand the links between practices and perceptions related to using interim 
assessment data and student achievement. Our analysis focused on the relationships between the four key dimensions of 
data use and student achievement on state assessments in reading and mathematics. We also examined the li nk s between 
perceived Barriers to Data Use and student achievement in both subjects. The analyses were conducted using a multi-level 
framework, meaning that we examined relationships among the data-use practices at both the school and classroom levels. 
This study was designed to address two broad research questions about using student data from interim assessments. 

Research Questions 

1. What are the relationships between teachers’ data-use practices and perceptions and their students’ 
achievement? 

2. What are the relationships between school policies, practices, and resources for data-driven instruction 
and student achievement? 

To test the hypothesized links in the theory of action, we measured teachers’ and principals’ data-use practices and percep- 
tions using surveys administered at three points during the 2009-10 school year. The surveys measured the key dimensions 
of data use among 1,581 teachers and 212 principals in a sample of 193 randomly selected elementary and middle schools 
in four geographically varied urban districts. 1 The analyses examined the links between the key dimensions and student 
achievement on state assessments in mathematics and reading. Test scores for 61,798 students were used in the analysis, and 
each student was linked to the principal and teachers responsible for their instruction. A detailed description of the method- 
ology is included in the main report. 


1 See Charting Success: Data Use and Student Achievement in Urban Schools , pgs. 83-84, for more information on the number of participating principals, teachers, and 
students. 
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Key Findings 


Descriptive Statistics 

In general, scores on the key dimensions were higher in the elementary grades than in the middle grades. 

• Elementary grades reading teachers reported more positive perceptions about using data and higher 
levels of data-use practices than middle grades reading teachers, with significant differences found in 
three of the four key dimensions — Context, Supports for Data Use, and Working with Data. However, 
there were no significant differences in Instructional Responses or Barriers to Data Use. 

• Elementary grades mathematics teachers also reported higher levels of data-use practices than mid- 
dle grades mathematics teachers, with significant differences found for both Working with Data and 
Instructional Responses. There were no significant differences among elementary and middle grades 
mathematics teachers on Context, Supports for Data Use, or Barriers. 

There were similar differences between elementary and middle grades principals. 

• Elementary grades principals had higher scores than middle grades principals on Context and Sup- 
ports for Data Use when reporting on both reading and mathematics. Elementary grades principals also 
reported significantly fewer Barriers to using data in both subjects than did middle grades principals. 

The finding that data-use perceptions and practices were higher among educators in the elementary grades than those in the 
middle grades may be due in part to the fact that elementary grades teachers often teach all subjects, have fewer students in 
total, and interact with their students for more time during the school day than do middle grades teachers. This may promote 
a more supportive data culture in which teachers and administrators are more likely to work with the data and engage in 
data-driven decision making. More support may be necessary to facilitate higher levels of data use in middle grades in urban 
districts. 

In addition to observing descriptive patterns for the key dimensions, we also examined correlations among the data-use 
scales as measured by our surveys. As expected, the four key dimensions — Context, Supports for Data Use, Working with 
Data, and Instructional Responses — all were positively correlated for both teachers and principals in each grade level and 
subject. These scales were negatively correlated with Barriers, also expected. 

Teachers’ General Data Use and Student Achievement 

We used multilevel structural equation modeling (SEM) to examine relationships between teachers’ general data use and 
student achievement in each grade level and subject area. First we created a latent variable of General Data Use by statisti- 
cally combining the scale scores for each of the four key dimensions ( Context , Supports, Working with Data, and Instruc- 
tional Responses ). 2 Next we tested the path between teachers’ General Data Use and student achievement. 


2 Although observed variables are directly measured (such as with a survey, observation, or interview), latent variables represent underlying constructs that are measured 
using multiple observed variables. For example, a latent variable of socioeconomic status may be made up of the observed variables of education, income, and profes- 
sional status. In this case, we created one latent variable of “General Data Use” with the observed variables of Context, Supports for Data Use, Working with Data , and 
Instructional Responses. 
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The results indicate that teachers’ general data-use practices and perceptions are positively related to student achievement 
in elementary grades reading and middle grades mathematics. That is, teachers with higher levels of General Data Use (and 
perceptions of data supports) have students with higher state assessment scores in middle grades mathematics and elemen- 
tary grades reading. The magnitude of this relationship was modest, with effect sizes of 0.10 for middle grades mathematics 
and 0. 1 7 for elementary reading. Yet shifting a student’s test score by 0. 1 7 standard deviations could have a significant effect 
on his or her academic standing. For example, if a student who was at the 50th percentile at the end of Grade 3 had a fourth- 
grade teacher who was at the mean on General Data Use, that student would be at the 57th percentile at the end of Grade 4. 
This could be the difference between a student being categorized as below proficient and proficient on a state assessment. 
Also, if a student were in classrooms for multiple consecutive years with teachers who have strong data-use perceptions and 
practices, this positive advantage could be cumulative over time. 

There was no statistically significant relationship between teacher data use and student achievement in elementary grades 
mathematics or middle grades reading. These findings in combination partially support the theory of action. 

Links between Individual Dimensions of Teacher Data Use and Student Achievement 

We used Flierarchical Linear Modeling (HLM) analyses to examine the relationship between each key dimension and stu- 
dent achievement. For these analyses, we examined — separately — Context, Supports for Data Use, Barriers, and a variable 
named Attention to Data in the Classroom, which was a combination of Working with Data and Instructional Responses? 
Each analysis controlled for background characteristics of the students, teachers, and schools, including prior achievement. 
Results indicate that some of the teacher data-use dimensions were significantly related to student achievement. 

Specifically, in middle grades mathematics and elementary grades reading, Attention to Data in the Classroom was signifi- 
cantly and positively related to student achievement, such that the more teachers reported reviewing interim assessment data 
and responding instructionally, the higher their students’ achievement on the state test. There was also a significant negative 
relationship between Barriers to Data Use and student achievement in elementary mathematics. 

Principals’ General Data Use and Student Achievement 

To address the question about school-level data use, we examined links between principals’ data-use practices and percep- 
tions and achievement for students in their schools. As with the teacher-level analyses, we used SEM to create a latent 
measure of principals’ General Data Use and examine relationships between it and student achievement in each grade level 
and subject area. 

Results indicate that principals’ General Data Use was positively related to student achievement, but only in middle grades 
mathematics. The link between General Data Use and student achievement was not statistically significant in elementary 
mathematics, elementary reading, or middle grades reading. 


3 The scales Working with Data and Instructional Responses were particularly strongly related, with correlation coefficients greater than r =0.75 for the teacher samples 
and 0.80 for the principal samples. This observation led us to consider whether these aspects of data use — defined as distinct in the theory of action (Exhibit 1) — are, in 
fact, too similar to separate, at least as measured by the survey instrument we used. As a result, we combined Working with Data and Instructional Responses for some 
of the analyses. 
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Links between Each Key Dimension of Principal Data Use and Student Achievement 

We used HLM analyses to examine the relationship between each key dimension at the school (principal) level and student 
achievement. For these analyses, we examined — separately — Context, Supports for Data Use, Barriers, and a variable 
named Attention to Data in the School, which was a combination of Working with Data and Instructional Responses for 
principals. As with the teacher analyses, each of these analyses controlled for a number of background student-, teacher-, 
and school-level characteristics, including prior achievement at the school and student levels. 

Results indicate that principal perceptions of Supports for Data Use were positively related to student achievement in el- 
ementary mathematics and reading, and their Attention to Data in the School was positively related to student achievement 
in elementary mathematics. 

No other dimensions of principal data use, including Context and Barriers to Data Use, were significantly related to student 
achievement. 

Discussion 

Although there was some prior evidence that using periodic assessments (formative assessments, progress monitoring, and 
curriculum-based measurement) may be positively related to student achievement, research on interim assessment use is 
limited. At the same time, significant investment has been made in interim assessment systems in school districts across the 
country. There is a great need for information about whether and how general and specific aspects of teachers’ and princi- 
pals’ data-use practices and perceptions are linked to student achievement. This study attempted to fill this gap by measuring 
multiple aspects of interim assessment data use among teachers and principals, and empirically testing the links between key 
data-use practices and student achievement on end-of-year state assessments. 

Summary of Findings 

We hypothesized that general and specific data-use practices and perceptions would be positively related to student achieve- 
ment. The findings partially supported this hypothesis. For both teachers and principals, General Data Use was related to 
student achievement in some grade levels and subjects. As for more specific practices and perceptions, teachers’ Attention 
to Data in the Classroom, principals’ Attention to Data in the School and principals’ perceptions of Supports for Data Use 
were related to higher student achievement. In other words, the more that teachers and principals reported reviewing and 
analyzing student data and using this information to make instructional decisions, the higher their students’ achievement, at 
least in some grades and subjects. Moreover, for principals, the more they reported having support in the form of data infra- 
structure, adequate time for review and discussion of data, professional development, and the appropriate human resources, 
the higher their students’ achievement. Again, these results varied by grade and content area, with significant links observed 
in both elementary grades and middle grades, as well as in mathematics and reading. 

Study Limitations 

This study used rigorous statistical modeling to explore the relationships between the key dimensions of data use and student 
achievement, but it is important to note some limitations. The school-based samples of principals, teachers, and students 
were sufficiently large in size, but there were only four districts, which may limit the generalizability of the findings. In 
addition, the study relied on self-report survey data rather than observations of actual data use and instructional practices. 
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Moreover, the study was not designed to provide information about the implementation quality of the interim assessment 
process in the participating districts, nor does it provide information about the quality of the actual interim assessments used. 
It also does not identify whether certain practices within the key dimensions of data use are more promising than others. 
Future work should continue to refine the measurement strategy to allow for analyses of the links between more specific 
data-use practices and student achievement than could be tested in this foundational study. 

Finally, although this study provides evidence of a relationship between some key dimensions of data use and student 
achievement, it is essential to understand that no causal claims about the nature of these relationships can be made on the 
basis of this correlational study. Interim assessments and the use of their data are just one of a number of policies, practices, 
and interventions being implemented within schools and school districts. It is not our claim that supports for data use and 
attention to data use in the classroom or school directly cause improved student achievement in certain grades and subjects. 
Rather, this study provides foundational evidence that as some aspects of data use increase, so too does student achievement. 

Future Directions 

Future research can help provide additional evidence of whether and how interim assessments can be used as a tool to in- 
crease student achievement. The ultimate goal would be to develop a set of standards or strategies that districts and schools 
can use as a guide for effective data use. 

Self-report measures of data use have provided valuable information on how interim assessments are used at classroom and 
school levels in the selected urban districts. As already noted, these data could be further enhanced through observations 
of actual data-use practices. These could include observations of data meetings where staff discuss the results of interim 
assessments or plan instructional responses and professional development on data use. Deeper study could also include an 
examination of lesson plans that stem from a review of the interim assessment data, along with classroom observations to 
explore how these plans are implemented. 

As we learn more about specific data-use practices that are associated with student achievement, another next step is to 
develop a more refined theory of action. This could involve developing specific data-use interventions intended to improve 
data use and instructional responses. Further research can also test the impact of such interventions on student achievement 
and identify the key mechanisms through which data use affects student learning. 

Of the six significant relationships between data-use practices/perceptions and student achievement that emerged in this 
study’s analyses, five were found in the elementary grades. This finding suggests that something about the structure of the 
elementary grades may be more conducive to the successful use of interim assessments, compared with the middle grades. 

Further research is needed to identify key factors in the elementary grades that could be adopted or adapted in the middle 
grades to potentially increase the utility of interim assessments for older students. Similarly, further exploration can help 
achieve a better understanding of the data-use differences in mathematics and reading that were revealed in this study. 
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Conclusions 

This study sought to begin to understand the connection between different aspects of interim assessment data use and stu- 
dent achievement. Given that school districts and schools are facing significant budget challenges and must make important 
decisions about resource allocation, it is imperative that we identify the specific dimensions of data use that are most im- 
portant for improving student outcomes. Many school districts are increasingly using various types of assessments and data 
in an effort to engage in data-driven decision making. Although the study focused primarily on the use of data from interim 
assessments, some of the study results may extend beyond interim assessments to provide a glimpse into the overall data 
culture of participating districts. As such, this study provides a foundation for future exploration of the relationships between 
student achievement and other types of data that can be used for instructional and school improvement. 

Finally, these findings have implications for data-use policies and practices in school districts and schools. This is particu- 
larly important as the nation moves toward the Common Core State Standards and the assessment systems that will accom- 
pany them. The findings suggest that, at the very least, if schools adopt interim assessments to produce changes in student 
achievement, schools and districts should provide adequate support for using the data, and teachers should actively use data 
in the classroom — both by spending time individually and collaboratively reviewing the student data and by responding 
instructionally. Although these findings do not identify the specific aspects of each dimension that are most important, it 
appears that data use by principals, particularly in elementary school, may be as important as teacher data use. This is in 
line with the findings from our site visits (as well as prevailing wisdom) that suggest that leadership and support from the 
administration are critical. The findings of this three-year project revealed that schools are better able to work with data 
when they have the appropriate data infrastructure, organizational supports for the analysis and productive discussions about 
data, human resources (e.g., data coaches), and professional development. In addition, there are important uses for interim 
assessment data by stakeholders at all levels. These include use by district leaders to identify professional development 
needs and evaluate district initiatives, use by school leaders to develop and evaluate school and staff improvement plans, 
and perhaps most importantly, use by teachers to inform instructional strategies. It is there in the classroom that students’ 
needs are most effectively met. 
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About the Council of the Great City Schools 


The Council of the Great City Schools is a coalition of 67 of the nation’s largest urban public school systems. 
The organization’s Board of Directors is composed of the Superintendent, CEO or Chancellor of Schools, and 
one School Board member from each member city. An Executive Committee of 24 individuals, equally divided 
in number between Superintendents and School Board members, provides regular oversight of the 501 (c)(3) 
organization. The composition of the organization makes it the only independent national group representing the 
governing and administrative leadership of urban education and the only association whose sole purpose revolves 
around urban schooling. 

The mission of the Council is to advocate for urban public education and assist its members in their improvement 
and reform. The Council provides services to its members in the areas of legislation, research, communication, 
curriculum and instruction, and management. The group convenes two major conferences each year; conducts 
studies of urban school conditions and trends; and operates ongoing networks of senior school district managers 
with responsibilities for areas such as federal programs, operations finance, personnel, communications, research 
and technology. Finally, the organization informs the nation’s policymakers, the media, and the public of the suc- 
cesses and challenges of schools in the nation’s Great Cities. Urban school leaders from across the country use 
the organization as a source of information and an umbrella for their joint activities and concerns. The Council 
was founded in 1956 and incorporated in 1961, and has its headquarters in Washington, D.C. 
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